A quantitative occam's razor
نویسندگان
چکیده
منابع مشابه
A Quantitative Occam's Razor
Interpreting entropy as a prior probability suggests a universal but “purely empirical” measure of “goodness of fit”. This allows statistical techniques to be used in situations where the correct theory — and not just its parameters — is still unknown. As developed illustratively for least-squares nonlinear regression, the measure proves to be a transformation of the R statistic. Unlike the lat...
متن کاملOccam's Razor
The Bayesian paradigm apparently only sometimes gives rise to Occam’s Razor; at other times very large models perform well. We give simple examples of both kinds of behaviour. The two views are reconciled when measuring complexity of functions, rather than of the machinery used to implement them. We analyze the complexity of functions for some linear in the parameter models that are equivalent ...
متن کاملA note on Occam’s razor
(a) (b) Figure 1: (a): The likelihood p(D|θ,Mcomplex) has a higher maximum value than then simpler model, but the likelihood drops quickly as we move away from regions of high likelihood. (b): The likelihood p(D|θ,Msimple) has a lower maximum value than then complex model, but the likelihood changes less quickly as we move away from regions of high likelihood. The corresponding fraction of para...
متن کاملSharpening Occam's Razor
We provide a new representation-independent formulation of Occam’s razor theorem, based on Kolmogorov complexity. This new formulation allows us to: (i) Obtain better sample complexity than both length-based [4] and VC-based [3] versions of Occam’s razor theorem, in many applications; and (ii) Achieve a sharper reverse of Occam’s razor theorem than that of [5]. Specifically, we weaken the assum...
متن کاملA Syntactic Justification for Occam’s razor
Informally, Occam’s razor states, “Given two hypotheses which equally agree with the observed data, choose the simpler”, and has become a central guiding heuristic in the empirical sciences and in particular machine learning. We criticize previous arguments for the validity of Occam’s razor. The nature of hypotheses spaces is explored and we observe a correlation between the complexity of a con...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: International Journal of Theoretical Physics
سال: 1983
ISSN: 0020-7748,1572-9575
DOI: 10.1007/bf02080316